Conformance testing in systems with semicontrollable interfaces

In a conformance testing environment, an implementation under test (IUT) communicates with multiple entities. A tester may have differing degrees of control on the interactions between these entities and the IUT: directly controllable, semicontrollable, or uncontrollable. Semicontrollable inputs most likely render portions of an IUT untestable. In addition, multiple communicating entities may create race conditions during testing. This paper presents a test generation methodology for the systems where the semicontrollable inputs can be generated indirectly. The test sequences obtained from the converted graph fully utilize the semicontrollable inputs (where possible) while avoiding the race conditions. Although, for the most general case, the graph conversion results in an exponentially large number of nodes, practical considerations make the converted graph size feasible. This approach is used to generate tests for MIL-STD 188-220B. By applying the proposed graph conversion and the race condition elimination techniques, the number of testable state transitions increased from approximately 200 to over 700, which represents a coverage of 95% of the transitions defined in the specification.